Instead of specifying methods, Congress should make sure that evaluators are clear about the questions to be answered.
Figure 1 above identifies 10 kinds of evaluation activities. At least part of the charge that evaluations have been irrelevant to Congress's needs for information stems from the fact that Congress has often been interpreted to be calling for impact evaluation when in fact it desired only to know, say, how well a program was meeting its coverage requirements. A call for evaluation that does not specify what questions are being asked can lead to the mismatching of expectation and performance by Congress and the evaluators. While legislators might include the policy questions to be addressed directly in the legislative provisions for evaluation of a program, it may not always be possible to frame questions with sufficient specificity at the time evaluation provisions are being enacted, especially for new programs. In such cases, sufficient dialogue should take place between the legislators and the implementing agency and the evaluators to ensure that the evaluation will meet its intended objective (Berryman and Glennan 1980).
Congressional mandates for evaluation should also identify the audience that is to be served by the legislated evaluation: Congress beneficiaries such as parent or other interest groups, local program administrators, federal program administrators, and the like. The reasons for specifying audiences in any evaluation are discussed in greater detail in later chapters. The reason for including audience specification in this recommendation is that such specification will also sharpen the policy questions because different audiences tend to have different information needs.
Though we recommend that it be specific with respect to question and audience, legislative language regarding evaluation should refrain from specifying details of method (such as sampling procedure or use of control groups) or of measurement. These are matters requiring careful technical consideration of specific evaluation conditions and contexts and should be chosen only after adequate planning and the application of expert knowledge